Current medical imaging devices offer a considerable number of functions and accordingly a considerable number of complex operating elements, which a user may use to configure and realize these functions. For example, embodiments of imaging processes and thus the imaging parameters vary greatly depending on the area imaged and the diagnostic goal. While common imaging processes are already hard to operate, this problem grows more severe if the medical imaging device is used in sterile environments, (e.g., during a surgical intervention). For example, medical imaging devices having a c-arm and a mobile carrier for the c-arm may be used as interventional modalities. In such an environment, an operating device or system has to be provided which on the one hand allows ergonomic use of the complex functions, on the other hand fulfils the requirements for a sterile operation.
In this context, it has been proposed to use mixed reality smart glasses as an operating element of the user interface device. For example, German Patent Application DE 10 2017 202 517.4 (published as EP 3305232) proposes to have at least one pair of mixed reality smart glasses as part of the user interface device.
The mixed reality smart glasses are configured to display at least part of the imaging process relevant information and/or to receive at least part of the imaging process relevant user input. Such mixed reality smart glasses have already been proposed in the state of the art, are a type of head mounted display, and may also be termed augmented reality smart glasses. Such smart glasses are, for example, available as “HoloLens” (Microsoft) or “MetaVision”. These devices may be worn covering the eyes of the user. An augmented reality is provided by projecting additional computer-generated information into the field of view of the user. Such a pair of smart glasses may include a plurality of sensors, in particular 3D and/or RGB cameras and/or at least one movement sensor for tracking the head wearing the smart glasses. The pair of smart glasses additionally includes projection units for each eye providing stereoscopic view, a control device, and/or a speaker/microphone. The control unit of the smart glasses may be configured to map augmented reality objects to the, in particular, sensed environment geometry and to display perspectively correct and stereoscopic graphical elements (e.g., augmented reality objects) using the projection units. The control unit may further be configured to detect and identify operating gestures performed by the user. It is noted that the control unit or at least parts of the control unit may also be located externally to the smart glasses, (e.g., inside a computing device). This is also possible for sensors. For example, an “inside-out” tracking of the smart glasses may be supported or replaced by external tracking devices (“outside-in” tracking), for example, by using externally trackable markers on the smart glasses. This may improve accuracy of tracking.
It has also been proposed to have more than one user wearing such smart glasses. These pairs of smart glasses may communicate directly or indirectly, allowing to match the augmented realities created for each user. Thus, all users see the same objects/graphical elements, configured to their respective point of view.
German Patent Application DE 10 2017 202 517.4 (published as EP 3305232) proposes to use such a pair of smart glasses to improve imaging processes in view of operation of the medical imaging device as well as informing the user.
A general problem in imaging systems is the presence of multiple moving components to establish different imaging geometries. An example is, again, a c-arm device. To provide the optimal medical information, imaging directions have to be chosen carefully. A further problem, in particular in 3D imaging, is the orientation of views of three-dimensional image data sets on corresponding display devices, (e.g., monitors). In certain systems, the view may be rotated by using a mouse or touchscreen, but this takes a long time to find the optimal direction. It has been proposed to use algorithms in medical imaging devices, which may automatically detect instruments or implants used in minimally invasive interventions and choose display orientations of three-dimensional image data sets accordingly. However, these methods may fail when used in workflows for which they have not been optimized or designed, for example, choosing a viewing direction when instruments or implants are not yet visible in image data.